Exploiting both local and global constraints for multi-span statistical language modeling

A new framework is proposed to integrate the various constraints, both local and global, that are present in the language. Local constraints are captured via ngram language modeling, while global constraints are taken into account through the use of latent semantic analysis. An integrative formulation is derived for the combination of these two paradigms, resulting in several families of multi-span language models for large vocabulary speech recognition. Because of the inherent complementarity in the two types of constraints, the performance of the integrated language models, as measured by perplexity, compares favorably with the corresponding n-gram performance.